CRISP-DM: Towards a Standard Process Model for Data Mining

The CRISP-DM (CRoss Industry Standard Process for Data Mining) project proposed a comprehensive process model for carrying out data mining projects. The process model is independent of both the industry sector and the technology used. In this paper we argue in favor of a standard process model for data mining and report some experiences with the CRISP-DM process model in practice. We applied and tested the CRISP-DM methodology in a response modeling application project. The final goal of the project was to specify a process which can be reliably and efficiently repeated by different people and adapted to different situations. The initial projects were performed by experienced data mining people; future projects are to be performed by people with lower technical skills and with very little time to experiment with different approaches. It turned out, that the CRISP-DM methodology with its distinction of generic and specialized process models provides both the structure and the flexibility necessary to suit the needs of both groups. The generic CRISP-DM process model is useful for planning, communication within and outside the project team, and documentation. The generic check-lists are helpful even for experienced people. The generic process model provides an excellent foundation for developing a specialized process model which prescribes the steps to be taken in detail and which gives practical advice for all these steps.